Region-Based Top-Down Segmentation Controlled by Stereo Matching

This paper describes experimental algorithms developed to investigate the problems of region based segmentation and matching processes on black and white and color stereo pairs of images. Our goal is twofolds: make these operations as automatic as possible and have them cooperate with each other. A cooperative automatic top-down segmentation algorithm is proposed. The two segmentation trees are constructed progressively: regions are validated if a satisfying match is found, and then their division is stopped in the trees. The algorithm is data driven and produces usable output for the matching and 3-D reconstruction processes even from the first division steps. intensity threshold s* is computed and must maximise a function AVCT(s) = RC(s)/N(s) which represents the average relative contrast observed on the edges detected by a threshold s [Koh81]. is the cumulated relative contrast on the detected edges and N(s) is the number of edges detected by s . The contrast-based approach produces accurate boundaries. The algorithm constructs contour-like regions, which are constructed around "solid" regions. Results were compared favorably with the contours produced by a Canny-like edge detector. A noise cleaning operation is integrated into the segmentation process. It merges "tiny" regions into their INTRODUCTION spatial context and may lead to the merge of bigger ones. Both segmentation speed and similarity are improved. The presented algorithms participate in a tasks sequence Only two image scannings are necessary for each image for stereo vision where a fundamental task to solve is division. The algorithm is data driven and produces the visual discrimination of objects and their location usable output for the matching and 3-D reconstruction in space. To achieve this goal, the main steps are SUCprocesses even at the first division steps. cessively: segmentation of a stereo pair of images, stereo matching of the regions, extracted in the 2 images, and the 3-D reconstruction of the facets corresponding to the matched regions. It is currently admitted that no method for deriving the "perfect" segmentation exists. The basic tenet of our approach is thus twofolds: The success of each job is closely connected to the results of the previous one, [RGSla]. The evaluation and improvement of segmentation relies on the goal specification of the algorithm. Therefore a way for us to evaluate and validate automatically the segmentation algorithm is to use its primary goal, which, in our case, is stereo matching. TOP-DOWN SEGMENTATION The segmentation algorithm uses an existing thresholding technique which relies on the notion of contrast rather than homogeneity. We have developed a region recursive multi-channel version. On each intensity channel of each dividable region, an SEGMENTATION RESULTS Figure 2 shows segmentation results on an artificial colour office scene named Officel, see figure 1. Figure 4 shows results and on a natural grey level office scene named Office2, see figure 3. The first division step thresholds one single region which is the whole image. Regions whose size is less than 4 pixels have been merged in their spatial context. SEGMENTATION DRIVEN BY STEREO MATCHING Another fundamental advantage of the top-down approach is that image divisions may be alternated with region matching, allowing a direct cooperation between these two processes. For segmentation, we use a goal oriented local validation criterion which is: a left region and a right region are well segmented if they match and if this match is valid [Ran92]. Since there is generally no optimal segmentation level, we have chosed a hierarchical approach, where all the already produced segmentation levels are stored, and the Figure 1: The original stereo pair of artzjcial colour Figure 3: The original stereo pair of natural images Ofimages Officel fice2 24 regions 13 regions ;I 101 regions 111 regions Figure 2: 2nd and 4th division steps on the pair Officel final regions of each image, are selected at different levels. The matching process drives directly the segmentation in the following way: matches are searched after each image division, and across the different already produced levels of segmentation, see figure 5. Thus, segmentation is driven by its immediate goal, which becomes its main convergence criterion. The final segmentation is derived from the validated matches rather than the contrary. STEREO MATCHING WITH BACKTRACKING The region matching technique uses an epipolar constraint and the attribute similarity of the regions to be matched. Color and boundary regularity are very important. The cooperative algorithm is top-down and automatic. The search for matches begins at the coarsest segmentation level. So, there is a risk for undersegmented regions to be similar enough to be matched, see 601 regions 588 regions Figure 4: 4th division step on the pair Office2 hgure ti. In order to avoid the selection of non coherent regions, correspondences found a t a given segmentation level may be reconsidered during a fixed number A, of later segmentation levels. The substitution of matches is performed according to the following rule: Let p~ = ( R I ~ ~ , h k r ) be a region pair found a t the iteration N = max(k1, kr) of the cooperative segmentation algorithm. If Rlk' or hk' has at least one descendent performing a better match Then p~ will be replaced by all its matched descendents in the match image. In order to control the matching process, a L1 planar analytical approximation [Abd75] of the intensity function of the matched regions is computed. The approximation error is used to detect under-segmentations and helps to control the validation of the performed matches. The bactracking on the matches will only start if the following constraint CApp(Rl, &) is verified. where N R ~ ( P ~ ) and NR(pe) are the number of nonisolated error pixels in each region Rl and &. Due to the results produced by the segmentation method, one bactracking level A, = 1 is mostly sufficient. L.ft . g . . t . t i o . . 9 . n t i 0 .